Generalization of the Second Law for a Nonequilibrium Initial State 
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We generalize the second law of thermodynamics in its maximum work formulation for a nonequi- 
librium initial distribution. It is found that in an isothermal process, the Boltzmann relative entropy 
(H-function) is not just a Lyapunov function but also tells us the maximum work that may be gained 
from a nonequilibrium initial state. The generalized second law also gives a fundamental relation 
between work and information. It is valid even for a small Hamiltonian system not in contact with 
a heat reservoir but with an effective temperature determined by the isentropic condition. Our 
relation can be tested in the Szilard engine, which will be realized in the laboratory. 
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The maximum work formulation of the second law of 
thermodynamics relates the work needed to move a sys- 
tem from one equilibrium state to another to the free 
energy difference between those states. It tells us that 
the work must be greater than or equal to the differ- 
ence in free energies. In recent refinements of the second 
law, such as the fluctuation theorem[I[ and the Jarzyn- 
ski equalityQQ, the maximum work formulation in the 
sense of the average work is rigorously shown for an ini- 
tial canonical distribution 0]. 

In this letter we generalize the maximum work formu- 
lation of the second law to transitions between nonequi- 
librium states. Our generalization allows one to find the 
maximum work that can be gained from such a transition 
and the processes that realize it. We are able to consider 
isolated systems as well as those coupled to a heat reser- 
voir and both adiabatic and isothermal transitions. 

The derivation is based on the Jarzynski equality mod- 
ified for a nonequilibrium initial distribution. This leads 
to a relation between the work and the Boltzmann rel- 
ative entropy with an effective temperature. The Boltz- 
mann relative entropy, also known as the Kullback- 
Leibler divergence, is always positive and gives a "dis- 
tance" between the nonequilibrium initial distribution 
and the canonical distribution [J]. 

For a finite Hamiltonian system without a heat reser- 
voir, the effective temperature is determined by an isen- 
tropic condition. The maximum work is realized in two 
successive processes: an instantaneous stabilization of 
the nonequilibrium initial distribution and an isentropic 
process. 

When the system is coupled to a large heat reservoir, 
the effective temperature is the temperature of the heat 
reservoir. From the generalized second law, the max- 
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imum work is realized in two successive processes: an 
instantaneous stabilization and a quasi-static isothermal 
process. For an isolated system the maximum work is re- 
alized by an initial stabilization followed by an isentropic 
process. 

The generalized second law gives a fundamental re- 
lation between work and information like the Landauer 
principle [5(. Actually, we are able to derive the Lan- 
dauer principle by applying our generalized maximum 
work formulation to the Szilard engineQQQ. We will 
also derive the Sagawa-Ueda generalization of the second 
lawQ with the Maxwell demon[l(| from our generaliza- 
tion. Our formulation would play an important role in an 
experimental realization of informational cooling as first 
proposed by Leo Szilardjo^flll]. 

In this letter, we consider a Hamiltonian system. The 
total system is divided into two parts, a small system 
and a large heat reservoir. X = (x, y) is a phase space 
point of the total system where x (y) is a point of the 
systemireservoir) . 

The total Hamiltonian is a sum of three Hamiltonians, 
the system, H^- S \x,a(t)), the reservoir, HW(y), and the 
interaction between them, H^(x,y), as 

H(X, a(t)) = H( s \x, a(t)) + H< R \y) + H^(x, y) (1) 

where a(t) is a time-dependent parameter associated with 
external operations. We operate on or observe only the 
system and not the reservoir. We assume no transition 
of particles (variables) between the system and the reser- 
voir. Hereafter, we represent a function in the system 
with the superscript, (S). 

The external operations are given by changing the 
external parameter, a(t), following a given protocol. 
For convenience, we assume that the external parame- 
ter is constant within unit intervals and instantaneously 
changes at integer times. That is, a(t) = a T for 
t e [r, r + 1) (t — 0, 1, 2, T). Hereafter, we abbreviate 
H T (X) = H(X,a T ) for convenience. 

A nonequilibrium distribution, po(X), is prepared at 
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t = 0. The time evolution of the probability distribu- 
tion of the total system is governed by the Hamiltonian 
dynamics as 

Pr(X) = C/ T _lC/ T _2 • • • U OP o(X) (2) 

where U T = exp(— iL r ) is the time evolution operator and 
L T is the Liouvillian corresponding to the Hamiltonian 
H r . 

The canonical distribution for the Hamiltonian H T (X) 
is defined as 

Pcan,r(^, a) = exp{a(F T (a) - H T (X))} (3) 

where a is a parameter like an inverse temperature and 
the free energy is F T (a) = — log{< l|exp(— aH T ) >}/a 
where < A\B >= J A*(X)B(X)dX in the bra-ket nota- 
tion. The canonical distribution is invariant, U T p caxl>T = 
Pcan,T, for any a, since it is a function of H T (X). 

The work is given as the sum of contributions from T 
phase points at t = r (r = 1,2, ...,T) on a trajectory 
starting at X , {X T (X n )}, 

T 

W(X ) = ^{ff T (X T (X )) - H T ^(X T (X ))}. (4) 

T=l 

The expectation value of the work is given as 

< W >=< W\p > . (5) 

It is important to note that "the maximum work" done 
by the system is the lowest work done on the system in 
our definition. Hereafter we will discuss a lower bound 
of the expectation value of the work. 

We generalize the maximum work formulation for a 
nonequilibrium initial distribution by starting with the 
Jarzynski equality 

< exp{a(AF(a) - W)}\p can , (a) >= 1 (6) 

where AF(a) = Ft (a) — Fo(a). For an invertible 
nonequilibrium initial distribution, we can modify it as 

< exp{a(A^(a) -W)- log( P ° - . )}\p Q >= 1 (7) 

Pam,0\ a ) 

By applying the Jensen inequality, < e~ x > > e~ <x> , 
we obtain the following important inequality 

<W> > AF(a) - — D(p |Pcan,o(a)) (8) 
a 

where D(p A \p B ) =< log(p A ) - \og(p B )\p A > is the non- 
negative Kullback-Leibler divergence, also known as the 
Boltzmann relative entropy, which gives the "distance" 
between two probability distributions, p A and ps- We 
note that it is valid for any a, since the the Jarzynski 
equality is based on the invariance of the canonical dis- 
tribution. 

We determine an effective temperature as the a" 1 , 
which maximizes the lower bound, 



Max Q [AF(a) — iD(po|Pcan,o(o ; ))]. The maximization 
condition requires the following isentropic property, 

< l0g(p C an,T(/3))|pcan,T(/3) > = < log(/9 )|p0 > (9) 

We first consider an isolated Hamiltonian system with- 

(S) (S) 

out the reservoir. Then, H T = Hi- and po — Pq ■ The 
generalized maximum work formulation for a nonequi- 
librium initial distribution in an adiabatic process is ob- 
tained with the effective temperature, 

<w> > < 4 5 VSt(/3) > - < ^ S U 5) > • 

.( 10 ) 

We show by what processes the maximum work is re- 
alized in the isolated Hamiltonian system without the 

(S) 

reservoir. We formally introduce a Hamiltonian, Ti. Q , 
for which the nonequilibrium initial distribution can be 
written as a canonical one, Pq 5 ' = exp(/3(J r g S '^(/3)— Hq S ^)) 

where (/3) is the free energy for H [ S) . Then, Eq.(10) 
is rewritten as 

< W > > < W >i S + < W >i E (11) 

where 

<^> IS ee <HF ) -H< 8) \fk 8) > (12) 

< W >ie ee < 4 S) Ip^,t(^) > - < ^ 5 U 5) > .(13) 

The most efficient way to gain work from the nonequilib- 
rium initial distribution in an adiabatic process is as fol- 
lows: (1) Change the initial Hamiltonian, Hq , to 7i a 

for instantaneous stabilization (IS) of the initial distribu- 

{ S) { s) 

tion. (2) Change Hq to the final one, H T in an isen- 
tropic process (IE). Here we assumed T is large enough 
and the isentropic process from p^ to p^fl T (/3) exists. 

Now we couple the system to the reservoir. We prepare 
the initial distribution separated from the reservoir at 
t = 0. Then we put the system in contact with the 
reservoir for intermediate times. Finally, we take the 
system away from the reservoir at t = T. We assume 
localized distributions for the both system and reservoir 
and a short range interaction. 

We assume that the reservoir is in equilibrium with 
temperature /? _1 at t = 0. Then, the total initial proba- 
bility distribution is given as 

po - exp{/3(F(«)(/3) - HW(y))}pW(x) (14) 

where F^ (/?) is the free energy of the reservoir. 

The effective temperature is determined by the max- 
imization of the right hand side of Eq.(8), after substi- 
tuting Eq.(14) into Eq.(8). After careful estimation of 
the cancellation of two large quantities in the Kullback- 
Leibler divergence, the effective temperature becomes the 
temperature of the reservoir. In the estimation, we as- 
sumed the energy of the system and the work are negli- 
gible compared with the energy of the reservoir. 
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The generalized maximum work formulation for a 
nonequilibrium initial distribution in an isothermal pro- 
cess is obtained as 

<W> >AF( s )(/3)-iD(p( s VSo(^)) (15) 

where we neglect the interaction at the both t = and t = 
T. The Boltzmann relative entropy (H- function) , with 
the temperature of the reservoir, has not only meaning 
as a Lyapunov function but also tells us the maximum 
work that may be gained from the nonequilibrium initial 
state. 

We show by what processes the maximum work is re- 
alized in the Hamiltonian system with the reservoir. We 
can rewrite Eq.(15) as 

< W > > < W >i S + < W >qi (16) 

where 

<^> IS = <n { s) ~H { s) \ P i s) > (i7) 

<W> Q1 ee 4 s) (/3)-^ 5) (/3) (18) 

where p^ S) = exp{/3(J^ S) (/?) - H^)}. Similar to the 
case of an adiabatic process, the most efficient way to 

gain work in an isothermal process is as follows: (1) 

(s) (S) 

Change the initial Hamiltonian, Hq , to Ti Q for the 
instantaneous stabilization (IS). (2) Change to the 

final one, Hf, , in a quasi-static isothermal process (QI). 
Here we assumed T is large enough and the quasi-static 
isothermal process from p^ to T (/3) exists. 

The Clausius relation in an isothermal process is also 
generalized. From the first law of thermodynamics, The 
absorbed heat is given as 

< Q > ee - < W > + < Ht\p { t ] > - < H \p { s) > 

(19) 

where p^\x) = J dypT{x,y). Then, the maximum ab- 
sorbed heat is obtained from the maximum work as 

<Q max > ee -/r 1 <log(^ T (/3))|^ T (/3)> 
+ r 1 <logU S) )U S) > (20) 

In the generalized Clausius relation, the maximum ab- 
sorbed heat in an isothermal process is given as the dif- 
ference between the final thermodynamic entropy and the 
initial Shannon entropy [3]. 

Here we mention a further generalization of the max- 
imum work formulation for a certain set of trajecto- 
ries (paths for a quantum system), which start from 
a nonequilibrium initial distribution and reach a final 
observable (detector). It is important for experiments 
where only particles entering a detector are observed. 
This can be modeled by choosing the final observable, 
A(Xt), which projects out a nonequilibrium final state. 
This generalization gives us the maximum work for a 



transition between two nonequilibrium states. The gen- 
eralization is straightforward using < A\p can ^T > instead 
of < l|/9 C an,T > in derivation of the modified Jarzynski 
equality. 

The generalized maximum work formulation of ther- 
modynamics for a nonequilibrium initial distribution 
gives an important relation between two important con- 
cepts in physics, the energy and the information. It is 
also a generalization of the Landauer principle When 
we apply the generalized maximum work formulation to 
the Szilard engine [6] 8], the famous relation, 

<W> >-ilog(2), (21) 

is immediately obtained, since the height of the one sided 
initial distribution, which has one bit of information, is 
just twice of the right-left symmetric canonical one in the 
double-well potential. 

Recently Sagawa and Ueda have generalized the Lan- 
dauer principle for quantum measurement Q . They 
showed how the maximum work is changed by measuring 
a quantum system as the Maxwell demon One of their 
important results is that the maximum work is given as 
the Shannon entropy with temperature in the most effi- 
cient classical measurement. We will derive it from our 
generalized second law to demonstrate its validity. 

Suppose the density matrix of a quantum system is 
the canonical one with the temperature, , before the 
measurement. It is given as 

Pcan =^2r n (l3)R n (22) 

n 

where R n is the projection operator for the nth energy 
level, E n , and r ra (/3) = exp{fl(F(f3) — E n )} is the proba- 
bility to measure the system at the nth energy level. 

After the most efficient classical measurement, the sys- 
tem may be in the nth energy level, P n = R n , with the 
probability r n (f3). From our generalized second law, in 
an isothermal process the work is bounded as 

W n > -iTr[{log(P„)~log(Pcan(/3))}P„] = ^log(r n (/3)) 

(23) 

As Sagawa and Ueda established, the expectation value 
is given as the Shannon entropy, 

<W> > i^r n (/3)log(r„ (/?)). (24) 

n 

It is interesting to apply this argument for an adiabatic 
process. The effective temperature, Z?^ 1 , is determined 
by the following isentropic condition, 

Tr[log(P can (/5„))P can (/3„)] = Tr[log(P„)P„]. (25) 

The effective temperature becomes zero in the most effi- 
cient classical measurement. From our generalized maxi- 
mum work formulation, in an adiabatic process the work 
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from the P n is bounded as 

W„>V lim n{Pi)Ei-E n = E -E n . (26) 

As we expected, we can gain all energy in the most ef- 
ficient classical measurement. The expectation value of 
the work is given as 

<W> >E -J2r n (P)E n . (27) 

n 

The maximum work in an isothermal process is greater 
than the one of the corresponding adiabatic process with- 
out the reservoir. This is clear from the definition of the 
effective temperature. In the adiabatic process, we can 
gain only the energy of the system without heat from 
the reservoir. For both adiabatic and isothermal pro- 
cesses the maximum work decreases in a non-efficient 
measurement [l4| ■ 

The generalized second law we introduce in this letter 
gives a relation between energy and information that is 
applicable even outside of thermodynamics. In such con- 
texts the thermodynamic temperature is replaced by an 
effective temperature determined by an isentropic con- 
dition. In this way the second law becomes universal 
parallel to the first law, which is the universal law of en- 
ergy conservation. As Ilya Prigogine asserted, the second 
law plays an important role not only in large thermody- 
namic systems but in small systems, such as elementary 
particles. 

Our generalized second law is valid for any Markov 
process with the invariant canonical distribution. There- 



fore, it is applicable to a system that may be modeled as 
a stochastic process, such as the Langevin equation in an 
isothermal process. It is interesting to apply our maxi- 
mum work formulation to a noncquilibrium steady state 
studied in such stochastic systems [151] and make clear 
from this perspective how house-keeping heat appears. 
We would also consider a Carnot-like cycle operation by 
combining adiabatic processes in Hamiltonian dynamics 
and isothermal processes in stochastic models. 

The results given in this letter open new perspectives 
for addressing long-standing issues in the study of irre- 
versible processes. Among those issues are the role of 
dynamical instability such as chaos, the reconciliation of 
reversible Hamiltonian dynamics with the approach to 
equilibrium, and the definition of nonequilibrium thermo- 
dynamic entropy in a reversible Hamiltonian dynamics. 
These will be addressed in a forthcoming paper that will 
consider the generalized maximum work formulation in a 
chaotic Hamiltonian svstem[l6j. The important proper- 
ties of Hamiltonian dynamics such as time-reversal sym- 
metry, the Liouville law, and the invariance of the Shan- 
non entropy, which we did not consider in this letter, will 
be discussed there. 
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